We present NusaCrowd, a collaborative initiative to collect and unite existing resources for Indonesian languages, including opening access to previously non-public resources. Through this initiative, we have has brought together 137 datasets and 117 standardized data loaders. The quality of the datasets has been assessed manually and automatically, and their effectiveness has been demonstrated in multiple experiments. NusaCrowd's data collection enables the creation of the first zero-shot benchmarks for natural language understanding and generation in Indonesian and its local languages. Furthermore, NusaCrowd brings the creation of the first multilingual automatic speech recognition benchmark in Indonesian and its local languages. Our work is intended to help advance natural language processing research in under-represented languages.

Dialogue systems can leverage large pre-trained language models and knowledge to generate fluent and informative responses. However, these models are still prone to produce hallucinated responses not supported by the input source, which greatly hinders their application. The heterogeneity between external knowledge and dialogue context challenges representation learning and source integration, and further contributes to unfaithfulness. To handle this challenge and generate more faithful responses, this paper presents RHO ($\rho$) utilizing the representations of linked entities and relation predicates from a knowledge graph (KG). We propose (1) local knowledge grounding to combine textual embeddings with the corresponding KG embeddings; and (2) global knowledge grounding to equip RHO with multi-hop reasoning abilities via the attention mechanism. In addition, we devise a response re-ranking technique based on walks over KG sub-graphs for better conversational reasoning. Experimental results on OpenDialKG show that our approach significantly outperforms state-of-the-art methods on both automatic and human evaluation by a large margin, especially in hallucination reduction (17.54% in FeQA).

Developing robust and fair AI systems require datasets with comprehensive set of labels that can help ensure the validity and legitimacy of relevant measurements. Recent efforts, therefore, focus on collecting person-related datasets that have carefully selected labels, including sensitive characteristics, and consent forms in place to use those attributes for model testing and development. Responsible data collection involves several stages, including but not limited to determining use-case scenarios, selecting categories (annotations) such that the data are fit for the purpose of measuring algorithmic bias for subgroups and most importantly ensure that the selected categories/subcategories are robust to regional diversities and inclusive of as many subgroups as possible. Meta, in a continuation of our efforts to measure AI algorithmic bias and robustness (https://ai.facebook.com/blog/shedding-light-on-fairness-in-ai-with-a-new-data-set), is working on collecting a large consent-driven dataset with a comprehensive list of categories. This paper describes our proposed design of such categories and subcategories for Casual Conversations v2.

随着深度学习和智能车辆的兴起，智能助手已成为促进驾驶和提供额外功能的重要车内组件。车内智能助手应该能够处理一般以及与汽车相关的命令并执行相应的操作，从而简化驾驶并提高安全性。但是，在这个研究领域，大多数数据集都采用主要语言，例如英语和中文。对于低资源语言，存在一个巨大的数据稀缺问题，阻碍了对更广泛社区的研究和应用的发展。因此，至关重要的是，拥有更多的基准来提高认识并激发低资源语言的研究。为了减轻此问题，我们收集了一个新的数据集，即广东话音频 - 视听语音识别（CI-AVSR），以使用视频和音频数据在广东话中使用拼写语言识别。与此同时，我们提出了广东话音频的语音识别在车内命令，这是社区在车内场景下应对低资源语音识别的新挑战。

Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, and visual-language generation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.

随着深度学习和智能车辆的兴起，智能助手已成为促进驾驶和提供额外功能的基本内部组件。汽车智能助理应该能够处理一般的和与汽车有关的命令，并执行相应的操作，减轻驾驶和提高安全性。但是，对于低资源语言存在数据稀缺问题，妨碍了研究和应用的发展。在本文中，我们介绍了一个新的DataSet，粤式视听语音识别（CI-AVSR），用于粤语中的车载命令识别，具有视频和音频数据。它由令人宣传的30个粤语发言者记录的200个车载命令的4,984个样本（8.3小时）组成。此外，我们使用常见的内部内部背景噪声增强我们的数据集来模拟真实环境，产生比收集的数据集大10倍。我们提供我们数据集的清洁和增强版本的详细统计信息。此外，我们实施了两个多模式基线以证明CI-AVSR的有效性。实验结果表明，利用视觉信号提高了模型的整体性能。虽然我们的最佳模型可以在清洁测试集上实现相当大的质量，但嘈杂数据的语音识别质量仍然是较差的，并且仍然是真正的车载语音识别系统的极其具有挑战性的任务。数据集和代码将在https://github.com/hltchkust/ci-avsr发布。

低资源语言的自动语音识别（ASR）改善了语言少数群体的访问，以便人工智能（AI）提供的技术优势。在本文中，我们通过创建一个新的粤语数据集来解决香港广东语言的数据稀缺问题。我们的数据集多域粤语语料库（MDCC）由73.6小时的清洁阅读语音与成绩单配对，从香港的粤语有声读物收集。它结合了哲学，政治，教育，文化，生活方式和家庭领域，涵盖了广泛的主题。我们还查看所有现有的粤语数据集，并在两个最大的数据集（MDCC和公共语音ZH-HK）上执行实验。我们根据其语音类型，数据源，总大小和可用性分析现有数据集。使用Fairseq S2T变压器，最先进的ASR模型进行实验结果，显示了我们数据集的有效性。此外，我们通过在MDCC和常见的声音ZH-HK上应用多数据集学习来创建一个强大而强大的粤语ASR模型。

代码切换是在对话期间交换语言时的语音现象。尽管对会话语言中的代码切换的自发性，但大多数现有工程通过读取语音而不是自发的语音来收集代码切换数据。Ascend（一个自发的中国英语数据集）介绍了香港收集的自发多转对话对话中英语代码切换语料库的高质量资源。我们报告了提升的设计和收集语音数据的程序，包括在这项工作中的注释。上升包括23个双语，这些双语流利，汉英都流利，而且由9.23小时的清洁语音组成。

In this work, we present a new computer vision task named video object of interest segmentation (VOIS). Given a video and a target image of interest, our objective is to simultaneously segment and track all objects in the video that are relevant to the target image. This problem combines the traditional video object segmentation task with an additional image indicating the content that users are concerned with. Since no existing dataset is perfectly suitable for this new task, we specifically construct a large-scale dataset called LiveVideos, which contains 2418 pairs of target images and live videos with instance-level annotations. In addition, we propose a transformer-based method for this task. We revisit Swin Transformer and design a dual-path structure to fuse video and image features. Then, a transformer decoder is employed to generate object proposals for segmentation and tracking from the fused features. Extensive experiments on LiveVideos dataset show the superiority of our proposed method.

运动转移旨在将驱动视频的运动转移到源图像。当驾驶视频中的对象与源图像中的对象之间存在很大差异时，传统的单个域运动转移方法通常会产生显着的伪影。例如，合成的图像可能无法保留源图像的人类形状（参见图1（a））。为了解决这个问题，在这项工作中，我们提出了一种运动和外观适应（MAA）进行跨域运动转移的方法，在该方法中，我们将合成图像中的对象正规化，以捕获驾驶框架中对象的运动，而仍保留对象在源图像中的形状和外观。一方面，考虑合成图像和驾驶框架的对象形状可能有所不同，我们设计了一个形状不变的运动适应模块，该模块可以在两个图像中强制对象零件的角度的一致性来捕获运动信息。另一方面，我们引入了一个结构引导的外观一致性模块，旨在使合成图像的相应贴片和源图像之间的相似性正式化，而不会影响合成图像中学习的运动。我们提出的MAA模型可以通过循环重建损失以端到端的方式进行训练，并最终产生令人满意的运动转移结果（参见图1（b））。我们在人类舞蹈数据集Mixamo-Video上进行了广泛的实验，以便于时尚视频和人脸数据集vox-celeb到cufs；在这两个方面，我们的MAA模型在定量和定性上都优于现有方法。